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A Study of the Institutional Factors Influencing the 
External Impacts of Student Assessment Information 



Abstract 

This study examines the external impacts and monitoring patterns of student assessment 
information on institutions based on institutional type and control in seven areas: student 
applications or acceptance rates, allocation or share of state funding, evaluation from regional 
accrediting agency, private fund-raising results, success on grant applications, communications 
with external constituents, and institutional reputation or image. It also examines how the 
external influences on, institutional approach to, institution-wide support for, and assessment 
management policies and practices designed to promote the use of student assessment 
information influence institutional monitoring of the external impacts of that information. Using 
discriminant analysis, the authors identify several conceptual domains and variables that can be 
used to discriminate between institutions that monitor the external impacts of student assessment 
information and those that do not. 
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A Study of the Institutional Factors Influencing the 
External Impacts of Student Assessment Information 

Problem and Research Question 

The issue of assessing student outcomes in higher education is of current interest in both 
education and government circles. The past few decades have seen increases in the demand for 
accountability in higher education, especially in public higher education. With this demand for 
accountability has come a greater scrutiny of how faculty and administrators assess the 
effectiveness of their respective institutions. This trend is not new (Peterson, Einarson, Trice, & 
Nichols, 1997). Typically, the field of higher education has seen a call for greater accountability 
and better assessment after periods of substantial growth. In this century, the periods from 1918- 
1928 and from 1952-1975 saw two periods of such expansion in higher education. Following 
each of these expansions were calls for increased assessment. In the earlier period concerns 
about coherence and substance in the curriculum spurred reform movements using 
“comprehensive testing” to assess college learning (Resnick & Goulden, 1987). As higher 
education experienced expansion in the 1950s and 1960s, much of which was brought on first by 
the GI Bill and then by attempts to increase access to higher education, the concern over quality 
resurfaced (Cameron & Whetton, 1996). One of the leading arguments against expanding access 
to higher education was that the result would inevitably decrease quality. Proponents of this 
argument felt that we could only increase access if we decreased admission standards. 

Opponents argue, however, that much of what limits access is the high cost of postsecondary 
education, which leads to another reason for the increase in accountability and assessment; the 
drastic rise in tuition dollars over the past two decades. The economic pressures that emerged in 
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the 1970s and 1980s put strains on institutional, state, and federal budgets, and led to public 
scrutiny over the use of tax dollars in higher education. These concerns triggered a greater 
interest in the accountability of higher education (Aper, 1993; Ewell, 1997). 

There are a number of reasons why assessment has reemerged as a critical topic in recent 
years. Aside from the strong political support from state officials who want to know where state 
funds are going and what they are buying, there has been a need to ensure that our local, 
regional, and state economies have a well trained workforce to meet the increasingly competitive 
challenges of a worldwide market (Rossmann & El-Khawas, 1987). Additionally, from higher 
education institutions, there has been a move toward improving instruction and curriculum, 
which has developed as a response to the reports that conclude higher education is not as 
effective as it could be (Erwin, 1991). Peter Ewell (1991) notes that the current movement in 
higher education has been prompted by a partnership between reform and accountability; “a 
partnership yielding a complex and diverse collection of assessment activities in university 
settings” (Riggs & Worthley, 1992). The net effect is that assessment is here to stay and many 
states and most institutions have initiated policies and programs to address the continuing 
concerns surrounding student outcomes (Peterson, Einarson, Augustine, & Vaughan, 1999). 

This recent fervor over outcomes assessment has lead to an ongoing national dialogue 
surrounding the issues of accountability and quality. Over the past decade, assessment and 
improvement of student performance has become the focus of much discussion and many efforts 
both within and external to colleges and universities. During that time there has been a 
progressive increase in the number of postsecondary institutions engaged in some form of 
student assessment (El-Khawas, 1990, 1995). To date, however, there has been little systematic 
examination of the external influences and demands for student assessment, the internal efforts* 
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and support for student assessment, and the uses and impacts of data from student assessment. 
One such study, the National Center for Postsecondary Improvement (NCPI) Research Project 
5.2 on “Institutional Support for Student Assessment” (ISSA) helped address this gap by 
examining the external influences on, institutional approaches to, institution-wide support for, 
management policies and practices of, and the uses and impacts of information from student 
assessment. 

Given the external sources for much of the demand for student assessment and 
accountability, one might expect that institutions would be interested in the impact that their 
student assessment efforts and information was having on certain external constituents. Several 
such impacts often referenced in the literature include the effect on: student applications, the 
institution’s share of state funding, the institution’s evaluation of regional accrediting agencies, 
private fund raising, the success of grant applications, communications with external 
constituents, and the institution’s reputation or image (Peterson et al, 1997). This association 
between institutional involvement in student assessment and the relationships with the external 
environment has received little consideration. The two areas most commonly referred to are 
state funding and accreditation (Banta, Lund, Black, & Oblander, 1996). No systematic 
empirical examinations of the relationship between institutional assessment efforts and external 
impacts has been located in the literature. One particular component of the NCPI study is the 
examination of whether institutions monitor the impacts derived from external sources, which 
are related to their student assessment practices, and if so, what characteristics distinguish those 
institutions from institutions that do not monitor these impacts. This is the basis for the research 
question to be addressed in this paper: what are the characteristic factors that distinguish 



institutions that monitor external impacts of student assessment information from those 
institutions that do not monitor the external impacts? 

Review of the Literature 

Prior to the collection of data for this study, an extensive literature review was completed 
on institutional support for student assessment (Peterson, Einarson, Trice, & Nichols, 1997). A 
total of 250 documents, most of which were descriptive or prescriptive in nature, were part of the 
initial review. Six distinct domains emerged from the literature: external influences on student 
assessment, institutional approaches to student assessment, institution-wide support for student 
assessment, assessment management policies and practices, institutional context, and 
institutional uses and impacts of student assessment. This study examines the influence of the 
first five of these domains on the external impacts of student assessment information. 

Additionally, several dimensions were identified within each domain. Of concern in this 
study are the following: accreditation and state influences (within the external influence domain); 
assessment content, assessment methods, and assessment studies (within institutional approaches 
domain); institutional support strategy, leadership and governance patterns, and evaluation of 
student assessment process (within institution-wide support domain); resource allocation, budget 
decisions, computer support, access to information, distribution of reports, student involvement, 
professional development, faculty evaluation, and academic planning and review (within 
assessment management policies and practices domain); institutional type and control (within 
institutional context domain); and external impacts (within the institutional uses and impacts 
domain). These dimensions will be discussed later in the variables section of the paper. 
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Nature of External Impacts of Student Assessment Information 

While there has been little consideration given to the external impacts of student 
assessment information, there are a number of areas cited in the literature that describe ways in 
which assessment information may be used for purpose external to the institution. The two most 
commonly cited uses of student assessment information is to respond to state and/or regional 
accrediting association reporting requirements (Banta et al., 1996; Banta & Moffett, 1987; 
Cowart, 1990; El-Khawas, 1989; Ory & Parker, 1989; Williford & Moden, 1993). El-Khawas 
(1995) reported in Campus Trends that 50% of the institutions she surveyed reported that student 
assessment was required by their state, 84% were required by their regional accrediting 
associations to include assessment in their self-studies, and 78% reported that specialized 
accrediting agencies required assessment to be included in their self-study. 

Banta et al. (1996) also cited that some institutions reported the use of assessment 
information in communications with local business and community leaders, other educational 
institutions, prospective parents and students, high school counselors, and potential donors. 
Depending on the audience, this use of assessment information may be intended to bolster the 
institutional image, illustrate public accountability, or create public awareness of institutional 
programs, services, or other related benefits (Banta & Moffett, 1987; Jacobi, Astin, & Ayala, 
1987). 

Institutional involvement in student assessment has been shown to provide some external 
impacts. There are a few institutions that have reported increases in institutional reputation or 
image as a result of their assessment activities and reported results (Young & Knight, 1993; 
Williford & Moden, 1993), increases in the number and academic potential of student applicants. 
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the receipt of awards associated with program excellence and academic challenge (Williford & 
Moden, 1993), and the allocation of state funding (McClain, Krueger, & Taylor, 1986). 

Apart from these few studies, very little attention has been given to the external impacts of 
assessment information on institutions. The two multi-institutional studies conducted in this area 
( Cowart, 1 990; El Khawas, 1 989) found that student assessment information was most likely to 
be used internally rather that externally. The literature found no research that examined the 
relationship between student assessment information and its external impacts (Peterson et al, 
1997). 

External Influences 

There are several areas that warrant consideration when examining external influences on 
student assessment; the federal government, state government, and regional accreditation 
associations being the most obvious. At the federal level, a number of reports, guidelines, goals, 
and regulations played key roles in promoting the current assessment environment (Peterson et 
al, 1997). Few empirical studies, however, have been undertaken to show the relationship 
between institutional assessment efforts and these federal initiatives. One area that has been 
studied in which the federal government did influence assessment is with the regional accrediting 
agencies. Muffo (1992) found that as a result of the stipulation from the U.S. Department of 
Education’s Criteria for Recognition, which states that accrediting agencies must require 
institutions to conduct student assessment, all six regional accrediting agencies now require the 
assessment of student outcomes. Additional federal influence was suggested by means of federal 
financial incentives (e.g. FIPSE, NSF) and several studies on assessment were funded with 
federal grants (Cook, 1989; Banta, 1991; Katz, 1993). 
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At the state level, the research by Cole, Nettles, and Sharp (1997) conducted from the 
National Center for Postsecondary Improvement, revealed that all but four of fifty responding 
states had reported some type of student assessment activity. Furthermore, wide variations in 
the approaches used by states have been documented throughout the literature. There are eight 
dimension, however, along which state assessment approaches can be compared (Peterson et al, 
1997). Four of these dimension are particularly interesting in examining the relationship of state 
influence to external impacts: state governance structure or authority (McGuiness, Epper, & 
Arredondo, 1994), types of state initiatives to student assessment (Cole et al, 1997; Sims, 1992), 
state purposes for assessment (Aper et al, 1990; Cole et al, 1997; McGuiness, 1994), and state 
approaches to student assessment (Aper et al, 1990; Banta & Moffett, 1987). 

The role of the accreditation associations has received less attention in the literature than 
state agents (Peterson et al, 1999). Still, these associations, particularly the regional accrediting 
associations appear to have a significant influence on the assessment efforts of institutions (Aper 
et al, 1990; Banta, 1993; Ewell, 1993). As discussed earlier, all six regional accrediting agencies 
now require student outcomes assessment as part of their institutional accreditation. 

Institutional Context 

The literature suggests that there may be distinct differences in the types of assessment 
initiatives and practices used by institutions based on institutional context. These differences 
have been found in the varying methods and approaches used by institutions and differ according 
to institutional type (Johnson, Prus, Anderson, & El Khawas, 1991; Steele & Lutz, 1995). Other 
studies have found that differences emerge in areas of organizational and administrative support 
for student assessment based on institutional type (Patton, Dasher-Alston, Rateray, & Kait, 1996) 
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and control (Johnson et al, 1991). These studies suggest the need to use institutional type and 
control when performing studies on student assessment impacts. 

Institutional Approaches 

Peterson et al (1999) identify institutional approach to student assessment as the content, 
timing, level of aggregation at which assessment occurs, and types of assessment measures used. 
The first of these, content, is identified in the literature using several taxonomies for classifying 
students’ performance or functioning (Astin, 1991; Bowen, 1977; Alexander & Stark, 1986; 
Ewell, 1984). Although these classifications use different terminology, all differentiate between 
various aspects of students’ cognitive (higher order processes, subject matter knowledge, etc.), 
affective (values, attitudes, satisfaction, etc.), and behavioral (course completion, hours spent 
studying, etc.) functioning (Peterson & Einarson, 1998). 

Timing refers to when and how often institutions choose to assess student outcomes. The 
literature identifies several stages at which institutions may collect student assessment 
information: prior to or at initial enrollment, during various points of enrollment, or near or after 
graduation, or any combination of these (Astin, 1991; Terenzini; 1989). Multiple point 
assessment is thought to have the greatest potential for impacting student and institutional 
performance (Astin, 1991; Jacobi et al, 1987). 

A third dimension of student assessment approaches is the level of aggregation to which 
the analysis or approach is oriented (Terenzini, 1989). There are a number of levels or units of 
analysis that an institution may choose to examine. They may use student assessment to study 
the performance of individual students, student subgroups, academic units or programs, schools 
or divisions or the entire institution (Alexander & Stark; 1986; Astin, 1991). 
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The fourth dimension examines the types of measures used in the assessment of student 
outcomes. This dimension includes not only the types of instruments used such as traditional 
pencil and paper tests or non-traditional portfolio analysis, but also includes whether the 
measures were developed internally or externally (Johnson, McCormick, Prus, & Rogers, 1993; 
Ory, 1991) or whether they are standardized across states or individual to the institution. 

These four dimensions would seem to be critical in examining the relationship between the 
student assessment approaches and the support for or uses of student assessment information, yet 
no research was found that used these dimensions to study the relationships. 

Institution-wide Support 

The literature discusses three dimensions of institution-wide support for student 
assessment: institutional support strategy, leadership and governance patterns, and the 
evaluation of the student assessment process (Peterson & Einarson, 1998). Institutional support 
strategy refers to the incorporation of assessment language and efforts into the mission, purpose, 
structure and function of the institution. Two distinct categories have been identified in the 
discussion of institutional support strategies : those that reflect an external focus, and those with 
an internal focus. Externally focused assessment support strategies underscore the adherence to 
external mandates such as state-level or accreditation requirements (Aper et al, 1990; Wolff & 
Harris, 1995). Internally focused assessment support strategies (Braskamp, 1991) emphasize 
institutional improvement efforts. 

The second dimension discussed in the literature is leadership and governance patterns. 
Leadership patterns refer to the critical role played by academic leadership in supporting the 
institutional efforts of assessment (Banta et al, 1996; Braskamp, 1991). This may take the form 
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of patterns of leadership participation, forms of support, or leadership styles. Banta (1993) has 
also suggested the importance of gaining formal and informal faculty and staff support. 

Governance patterns refer to the structures and processes used in institutional student 
assessment efforts (Peterson et al, 1999). These may vary from who and/or how many oversee 
the assessment process to the degree to which the authority for decision making is centralized or 
decentralized (Banta et al, 1996). Johnson et al (1991) examined the structures that institutions 
implement for student assessment. They found that executive responsibility was most often 
situated in the academic division of an institution, the operational responsibility was usually 
given to an administrator rather than faculty member, and that a minority of institutions had 
actually created separate offices for student assessment. 

The third dimension found in the literature is an evaluation component of the institution’s 
assessment process. According to Wolff (1992), when student assessment is approached as a 
constantly evolving process, it can contribute to the improvement of student and institutional 
performance. For this process to work, it needs to be evaluated on a regular basis. According to 
Patton et al (1996), few institutions evaluate their student assessment process and activities. 
There is little evidence in the literature, however, to support a relationship between student 
assessment evaluation and its impacts. 

Assessment Management Policies and Practices 

The final domain found in the literature discusses institution’s assessment management 

policies and practices. This refers to the extent to which the assessment process is translated into 

various institutional policies and become routine practice. This process is seen as critical in 

sustaining the effectiveness of the assessment process (Ewell, 1988). The literature describes a 
% 

number of practices that are recommended as institutions develop and implement their student 
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assessment program. These include: access to and distribution of reports (Patton et al, 1996), use 
in faculty evaluation to promote faculty involvement (Ewell, 1984; Ryan, 1993), professional 
development in assessment (Banta et al, 1996; Young & Knight, 1993), the involvement of 
students (Duvall, 1994) and administrators from student affairs (Erwin, 1991), and the use of 
assessment data in academic planning and review (Ewell, 1988). 

Only one study was located that addressed the relationship of assessment management 
practices and policies to external influences or to an institution’s assessment approaches, and 
internal support. This study found that institutional practices regarding the assessment reports 
was predictive of achieving positive outcomes for student assessment projects, but that using 
resource allocations was not (CSUITL, 1993). No other literature on this relationship was 
located. 

Conceptual Framework 

Based on the literature review, a conceptual framework of institutional factors influencing 
external impacts was developed and is comprised of six domains: external influences, 
institutional context, institutional approaches to student assessment, institution-wide support for 
student assessment, assessment management policies and practices, and external impacts of 
student assessment information. The framework is displayed in Figure 1. 



Research Questions 

The purpose of this study was to examine the influence of the five specific domains 
identified in the conceptual framework on the domain representing the external impacts of 
student assessment information. The five domains are external influences on student assessment, 
institutional approaches to student assessment, institution-wide support for student assessment,* 
management policies and practices for student assessment, and institutional context. 
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Figure 1 . Conceptual Framework for Institutional Factors Influencing External Impacts of 
Student Assessment Information 
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Additionally, we were interested in identifying any differences among and between 
institutions that might be used to identify whether or not these institutions were monitoring the 
external impacts of their assessment information. The research questions addressed in this study 
were: 

1 . What types of external impacts of student assessment information do institutions 
monitor? Do these vary by institutional type? Do these vary by institutional control? 

2. How do institutions’ external influences on, institutional approaches to, institution-wide 
support for, and patterns of assessment management policies and practices for student 
assessment relate to the external impacts of student assessment information they monitor? 
Does this vary by institutional type? 



These questions were the starting point for this study. Based on the literature review, the 
conceptual framework, and the research questions, we were able to identify a number of 
variables that we used in this study. These are described in the next section. 

Variables 

Each of the variables used in this study is listed below and the variable names are italicized 

for easy identification. The operational definitions of the variables can be found in Appendix A. 

External Influences. The variables in this domain include accreditation influence, which 

captures the influence from regional accrediting agencies, and state initiative, state approach, 

and state authority, which capture the influence of the state in which the institution resides. 

Institutional Context. This domain contains only two variables for this study, institutional 

type, (divided into five collapsed Carnegie types: associate of arts, baccalaureate, 

comprehensive, doctoral, and research) and institutional control (public or private). Since the 

variables in discriminant analysis should be continuous (or in the case of institutional type, can 

be forced continuous) institutional control is not used in the discriminant analysis. It should be 

noted, however, that there are differences by institutional control for three of the external 

impacts. For the second external impact, allocation or share of state funding, the difference in 

monitoring by control is expected since private institutions generally receive no stato funding. 

Because of this, private institutions were removed from the analysis of the second external 

impact, allocation or share of state funding, during our examination by institutional type. (See 

result’s section for tables and a more detailed discussion.) 

Institutional Approach to Student Assessment. Institutional approaches to student 

assessment are measured by the extent to which they collect data on students’ academic 

% 

intentions and in the areas of students’ cognitive assessment, affective assessment, and post- 
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college assessment. Furthermore, an additive index was created to capture the 
comprehensiveness of data collection in these various areas. In measuring the various methods 
used by institutions, three variables are used to capture institutional approaches, number of 
instruments, { the number of inventories or comprehensive tests used), student methods 
(integrative and performance based methods), and external methods (methods developed outside 
the institution or performed using external constituents). The final two variables in this domain, 
number of studies and number of reports capture the quantifiable analysis of the assessment 
approach by aggregating the total number of assessment studies conducted and the number of 
levels at which the data is reported, respectively. 

Institution-wide Support for Student Assessment. Seven variables represent the domain of 
institution-wide assessment support. The mission emphasis examines whether or not the 
. institution includes within its mission any mention or emphasis on undergraduate assessment. 
Internal purposes evaluates the importance of internal reasons for engaging in student 
assessment. Likewise, accreditation purposes, and state purposes measure the importance of 
accreditation standards and state requirements in the institutions’ assessment efforts. The type 
and amount of administrative and faculty support are captured in this variable, and the number 
and level of individuals involved in the institutions’ assessment group are captured by the 
breadth of assessment planning group. The final variable in this domain, number approving 
changes, identifies the number of individuals needed to change the assessment plan or process. 

Assessment Management Policies and Practices. Ten variables were developed relating to 
management policies and practices, seven of these were used in the analysis for this study: 
access to information measures the breadth of internal access to assessment reports; distribution 
of reports measures the breadth of report distribution in the institution; the extent to which 
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institutions use policies that promote student involvement in the assessment process; the use of 
professional development policies or practices to involve faculty and administrators in 
assessment; the use of professional development policies for administrators in student affairs', 
the use of student assessment data in the academic planning and review process; and the use of 
policies or practices tying faculty evaluation to the use of assessment information or 
involvement. 

As in the last domain, there were other variables developed in the larger study of 
institutional assessment. Due to the nature of this smaller study and the restrictions of the 
analytical method used, the other variables were not incorporated and were left unidentified to 
eliminate any confusion. 

External Impacts. Seven possible dimensions were described in the literature relating to 
external impacts of student assessment information. These include: student application or 
acceptance rates, allocation or share of state funding, evaluation from regional accrediting 
agency, private fund-raising results, success on grant applications, communications with 
external constituents, and institutional reputation or image. These seven dimensions were 
measured by the extent to which an institution monitored the influence of student assessment 
data on these areas. 

Method 



Survey Design 

This study is part of Project Five of the National Center for Postsecondary Improvement 
(NCPI). NCPI is a national research project funded by a grant from the Office of Educational 
Research and Improvement (OERI) of the U.S. Department of Education. The data used in this 
study were collected from a survey that was developed out of the literature review and during the 
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second phase of the four phase project. The design of the instrument used two approaches: the 
identification of dynamics, policies and practices reported in the literature, and the the use of 
instruments used in other surveys of student assessment. The instrument was designed based on 
these sources. The instrument, “Inventory of Institutional Support for Student Assessment” was 
piloted and refined using chief academic officers in four types of institutions (associate of arts, 
baccalaureate, comprehensive, and research). The survey is a comprehensive inventory of 
external influences of student assessment, institutional approaches to student assessment, 
organizational and administrative support for student assessment, and institutional uses and 
impacts of student assessment information. 

Population 

The survey was sent in January of 1 998 to the chief academic officers of all 2,524 two-year 
and four-year institutions (not including specialized or proprietary institutions) recognized by the 
U.S. Department of Education who offer undergraduate programs at the associate of arts or 
baccalaureate level. The administrators were asked to forward the survey to the person or 
persons most familiar with undergraduate student assessment at the institution. We received 
1,393 completed surveys for a response rate of 55 percent. The response rate by institutional 
type varied from 53% for baccalaureate institutions to 65% for research institutions. 

Of the 1393 institutions that responded, a large number did not respond to the specific 
items to be used as the grouping variable in the discriminant analysis. Therefore 221 institutions 
were eliminated in order to satisfy the statistical requirement of the analysis. This left a total of 
1 1 72 institutions in the final analysis. 
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Data Collection 



The data collection process included five steps. First, two weeks prior to sending the 
survey, a letter was sent to the chief academic officers to inform them of the nature and 
importance of the national study and to let them know that the survey would be arriving shortly. 
Second, the survey was then sent to the same administrator with an attached cover letter that 
discussed its intended use and directions for completing and returning it. Third, a week after the 
survey was sent, a follow-up post card was mailed to remind the administrator of the importance 
of completing and returning the survey. Fourth, approximately a month after the survey was 
sent, all non-responding institutions received a phone call encouraging them to respond and 
offering to answer any questions. Finally, a second mailing of the survey was sent to all non- 
responding institutions. 

As the returned surveys were received, they were recorded, reviewed for accuracy of 
responses, and checked to see if the IPEDS identifier was still attached. This would allow us to 
merge institutional characteristics from the national IPEDS database with our survey data. The 
surveys were then sent to a commercial data entry firm for computerized entry of the data. All 
entries were double verified and a complete computer disk was transferred to the research team. 
The team then performed a random check of computerized data against surveys and found no 
errors. The final step was to merge the computerized survey data with the IPEDS database. 
During the analysis (discussed in the next section), a set of derived indices and factor variables 
would be added to the database. 

Analysis 

The data were first analyzed using several traditional methods including descriptive 
analysis and frequency distributions. Means and standard deviations for all survey items were 
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reviewed to identify any inconsistencies among similar items or items for which there was little 
or no variation. The researchers used factor analysis to create indices for items thought to be 
related to common dimensions of the conceptual framework. Each factor analysis was rotated 
using the oblique option and items were included if they met three criteria: 1) were weighted 
most heavily on that factor, 2) had a loading exceeding .40, or 3) were conceptually similar in 
content to other items in the factor. Any items that did not load heavily were retained as 
individual items. Additionally, additive indices were created for items not subjected to factor 
analysis. This data reduction helped to reduce the overall number of variables and gave us a mix 
of individual items, factor indices, and additive indices 

The initial plan was to run multivariate regression analysis using items and indices to 
examine the relationships between and the influences of the five domains on the external impacts 
of institutions. However, it was determined that the proposed dependent variable (external 
impacts) was not continuous and therefore could not be used as an outcome measure in 
regression analysis. The researchers then decided to use discriminant analysis to identify any 
variables which might be used to discriminate between institutions that do monitor external 
impacts and those that do not. 

In order to facilitate the discriminant analysis, the researchers first recoded the seven items 
included as external impact variables. These were initially coded as follows: 0= not monitor; 

1= monitored, negative impact; 2 = monitored, no known impact; 3= monitored, positive 
impact. After recoding the items were coded as 0= did not monitor external impacts; 1= 
monitored external impacts. This created a dichotomous variable to be used in the discriminant 
analysis. 
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Since discriminant analysis is a listwise procedure (requires all cases to have full data on 
every item) and since we were missing data on a number of items for various cases, we needed to 
replace the missing data or eliminate the case. We used mean replacement for those variables 
with fewer than 10% of the cases missing data. For those with more than 10% missing we 
eliminated the variable from our analysis. 

Stepwise discriminant analysis (Nie, Hull, Jenkins, Steinbrenner, & Bent, 1975) were 
performed separately for each of the seven items in the external impacts domain. This allowed 
variables to enter the analysis until the variables remaining in the pool offered no significant 
increase in the discriminatory ability (significance of F to enter = .05). 

Discriminant analysis results will provide n-1 discriminating functions; where n= number 
of group variables. In this study, since we are looking only at whether institutions monitor their 
external impacts or not, there are only two group variables. Therefore, each discriminant 
analysis only produced one discriminating function for each of the seven external impact items. 
The discriminant analysis with corresponding eigenvalues, Wilks’ lambdas, group centroids, and 
standardized discriminant coefficients for each function are discussed in the results section. 

The eigenvalue by itself tells us little. It becomes more useful in distinguishing the strength 
of the function with respect to other functions. In this study, since there are only two groups 
(institutions that monitor and institutions that do not monitor) and thus one function, the 
eigenvalue provides little indication of a functions relative strength. It is included, however, 
because we can draw some comparisons of the relative strength of discriminant functions for 
each of the seven external impact items. Similarly, the Wilks’ lambda is a comparative measure 
and can be used to compare functions, but unlike the eigenvalue it does have meaning by itself. 
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Wilks’ lambda ranges from 0 to 1 and the lower the number the better the relationship between 
the function and the group membership. 

The group centroid is another means of distinguishing the function that discriminates best 
between those institutions that monitor external impacts and those that do not. The group 
centroid graphically defines each groups location along the functions continua between positive 
1 and negative 1 . In this study the positive number represents institutions that do monitor 
external impacts and the negative number represents institutions that do not monitor external 
impacts. 

The standardized discriminant coefficients help us interpret the function. They represent the 
influence of their respective discriminating variables on the overall function. The absolute value 
of the coefficient indicates the importance of that variable in the interpretation of the function 
and the sign indicates its direction toward the positive or negative end of the continuum of 
interpretive statements. In this case a positive sign would indicate a variable’s influence in the 
function toward differentiating an institution’s likelihood to monitor its external impacts and a 
negative sign would indicate a variables influence in the function toward differentiating an 
institution’s likelihood not to monitor its external impacts. 

Results 



The descriptive results show a weak pattern of monitoring the external impacts of student 
assessment information. The discriminant analysis, however, has shown that there are a number 
of significant discriminating variables that can be used to differentiate between institutions which 
monitor external impacts of student assessment information and those that do not. Though each 
of the seven external impacts had a discriminant function with a number of discriminating 
variables-some were stronger than others-each had its own unique discriminating variables. 
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Furthermore, we were able to identify a number of discriminating variables that contributed 
across several of the external impacts. These discriminating variables provide dimensions that 
can be used to differentiate between institutions that monitor versus those that do not, for three or 
more external impacts. We will first discuss descriptive results, then the discriminant functions 
for each of the seven external impacts highlighting the variables that are relevant to each and 
finally the variables by conceptual domains that are related to several of the external impacts. 

Descriptive Results 

A review of the descriptive pattern of institutional monitoring of the external impacts of 
student assessment information addresses our first research question: What types of external 
impacts of student assessment information do institutions monitor? Do these vary by institutional 
type? by institutional control? Table 1 shows the monitoring patterns by institutional type and 
Table 2 shows the monitoring patterns by institutional control. The results of the overall pattern of 
monitoring are shown in the “Total” row of Table 1 and / or Table 2. 

Overall Institutional Monitoring (Table 1, Total row) shows a low pattern of monitoring of all 
seven external impacts of student assessment information. With the exception of the external 
impact evaluation from regional accrediting agency, all of the impacts are monitored by less than 
one-third of the institutions. Monitoring of the external impact private fund-raising results was the 
lowest with only 15.9% of the institutions monitoring for this. The external impact on evaluation 
from regional accrediting agency was highest with 56.3% of the institutions monitoring this area. 
This may be some indication of the relative importance or influence of regional accrediting 
agencies in the collection of student assessment information. Monitoring for student assessment 
impacts on institutional reputation or image at 34.1% was the second highest, perhaps reflecting 
institution’s concerns in this dimension. 
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Table 1. Monitoring Differences by Institutional Type. (N=l 172) 



Institutional Type 


External Impacts 


Student 

Application 

or 

Acceptance 

Rates 


Allocation 
or Share of 
State 
Funding 1 


Evaluation 

from 

Regional 

Accrediting 

Agency 


Private 

Fund- 

Raising 

Results 


Success 
on Grant 
Applica- 
tions 


Communica- 
tions with 
External 
Constituents 


Institu- 
tional 
Reputation 
or Image 


Associate of Arts 
















Monitor 


16.6% 


22.4% 


54.0% 


11.0% 


26.0% 


24.6% 


33.2% 


Do Not Monitor 


83.4% 


77.6% 


46.0% 


89.0% 


74.0% 


75.4% 


66.8% 


Baccalaureate 
















Monitor 


25.3% 


33.3% 


56.4% 


23.4% 


27.1% 


29.7% 


37.0% 


Do Not Monitor 


74.7% 


66.7% 


43.6% 


76.6% 


72.9% 


70.3% 


63.0% 


Comprehensive 
















Monitor 


20.0% 


31.3% 


60.4% 


16.8% 


21.1% 


27.5% 


36.4% 


Do Not Monitor 


80.0% 


68.9% 


39.6% 


83.2% 


78.9% 


72.5% 


63.6% 


Doctoral 
















Monitor 


14.0% 


32.5% 


61.4% 


15.8% 


19.3% 


26.3% 


24.6% 


Do Not Monitor 


86.0% 


67.5% 


38.6% 


84.2% 


80.7% 


73.7% 


75.4% 


Research 
















Monitor 


19.4% 


37.8% 


51.6% 


17.7% 


16.1% 


32.3% 


27.4% 


Do Not Monitor 


80.6% 


62.2% 


48.4% 


82.3% 


83.9% 


67.7% 


72.6% 


Total 
















Monitor 


19.5% 


26.3% 


56.3% 


15.9% 


24.2% 


27.0% 


34.1% 


Do Not Monitor 


80.5% 


73.7% 


43.7% 


84.1% 


75.8% 


73.0% 


65.9% 


Chi-square 


9.62* 


10.32* 


4.12 


20.95*** 


6.58 


3.37 


5.41 



Note 1: N=753 Public Institutions 
* p < .05; ** p < .01; < .001 



Monitoring by Institutional type (Table 1) shows a significant difference in the monitoring 
patterns among types of institutions for three of the external impacts: student application or 
acceptance rates, allocation or share of state funding, and private fund-raising results. For 
student application or acceptance rates , baccalaureate institutions were highest with 25.3% 
monitoring this impact versus a low 14% of doctoral institutions that monitor this impact. Others 
fell between these extremes. This difference may be attributed to the relative importance of 
student applications and acceptance rates for various institutional types, their emphasis on 
undergraduate education, or the competitiveness of their market segment. 



Table 2. Monitoring Differences by Institutional Control. (N=l 172) 



Institutional Control 


External Impacts 


Student 

Application 

or 

Acceptance 

Rates 


Allocation 
or Share of 
State 
Funding 


Evaluation 

from 

Regional 

Accrediting 

Agency 


Private 

Fund- 

Raising 

Results 


Success on 
Grant 
Applica- 
tions 


Communica- 
tions with 
External 
Constituents 


Institu- 
tional 
Reputation 
or Image 


Public Institutions 
















Monitor 


19.0% 


26.3% 


58.0% 


13.8% 


24.2% 


28.7% 


36.4% 


Do Not Monitor 


81.0% 


73.7% 


42.0% 


86.3% 


75.8% 


71.3% 


63.6% 


Private Institutions 
















Monitor 


20.3% 


5.5% 


53.2% 


19.6% 


24.3% 


23.9% 


30.1% 


Do Not Monitor 


79.7% 


94.5% 


46.8% 


80.4% 


75.7% 


76.1% 


69.9% 


Total 
















Monitor 


19.5% 


18.9% 


56.3% 


15.9% 


24.2% 


27.0% 


34.1% 


Do Not Monitor 


80.5% 


81.1% 


43.7% 


84.1% 


75.8% 


73.0% 


65.9% 


Chi-square 


.288 


76.16*** 


2.53 


6.69** 


.004 


3.17 


4.77* 



* p < .05; ** p < .01; ***p £ .001 



For the external impact of student assessment on allocation or share of state funding, 
research institutions monitored at a high rate of 37.8% versus a low of 22.4% for associate of arts 
institutions. While it is risky to speculate as to the reason for these differences, it might be that 
research institutions rely very heavily on additional state funding relative to tuition and fees 
whereas associate of arts institutions may rely less on state assistance and more on funds 
generated by tuition and fees and local taxes. 

The differences in the monitoring rates for the external impact of student assessment on 
private fund-raising results can probably be attributed to the public / private control of 
institutions. The results show that a high of 23.4% of baccalaureate institutions monitor this 
external impact, versus a low of 1 1% of associate of arts institutions. The results of monitoring 
by institutional control (see the next section) show a significant difference in the monitoring 
practices of the same external impact, private fund-raising results, and that private institutions 
monitor this impact more that public institutions. Since a large number of private institutions are 
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classified as baccalaureate, it makes sense that the monitoring of this external impact would not 
only be significantly different by institutional control, but also by institutional type (i.e. a high 
correlation between private institutions and baccalaureate institutions). 

Monitoring by Institutional control (Table 2) shows a significant difference in the 
monitoring patterns between public and private institutions for three of the external impacts: 
allocation or share of state funding, private fund-raising results, and institutional reputation or 
image. Private institutions monitor the impact of student assessment on allocation or share of 
state funding less than public institutions (5.5% versus 26.3%). It is not surprising that this 
external impact is monitored differently by institutional control since private institutions 
generally receive limited funding from states. Correspondingly, private institutions monitor the 
external impacts of student assessment on private fund-raising results more than public 
institutions (19.6% versus 13.8%), most likely because they rely more heavily on private 
donations for their overall funding. Finally, and somewhat surprisingly, private institutions 
monitor the external impact on institutional reputation or image less than public institutions 
(30.1% versus 36.4%). This difference may be due to the general perception that private higher 
education is of a higher quality than public higher education and therefore, public institutions 
more closely monitor the external impacts of their student assessment information. 

Discriminant Functions 

As previously mentioned, we are only differentiating between two groups-institutions that 
monitor external impacts and those that do not. Discriminant analysis gives only one function 
for each impact. While discriminant analysis does not directly compare the relative strengths of 
different functions among the impacts, we can examine the relative strength by comparing each 
function against the others. This comparison is useful and can help us better understand the 



external impacts by examining the differing patterns of variables that discriminate on each 
impact and the domains that contribute to the functions across all of the impacts. This analysis 
allows us to address our second research question, “How do institutions’ external influences on, 
institutional approaches to, institution-wide support for, and patterns of assessment management 
policies and practices for student assessment relate to the external impacts of student assessment 
information they monitor?”, for each of the seven external impacts and provides the basis for 
further examination in the final results section. These results are shown in Table 3. 

The External Impacts of Student Assessment on Allocation or Share of State Funding , (Table 3) 
provides the best overall discriminant function. This is corroborated by the lowest Wilks’ 
lambda (.829) and the highest eigenvalue (.207). Additionally, the function at the group centoid 
shows the largest difference between groups (.761 and -.271) on the 1 to -1 continuum. Since 
private institutions rarely, if ever, receive state funding, this analysis was run using only public 
institutions. Another indicator of the relative strength of this function is the number of 
discriminating variables that emerge from the five domains; there are 12 discriminating variables 
that significantly contribute to the overall function and at least one from each of the five 
domains. The strongest of the discriminating variables, as one might expect for this external 
impact, is state initiative (.407). Two other relatively strong discriminating variables include 
institutional type (.381) and state purposes (.324). There are two variables in this function for 
which the influence is negative, accrediting influence and external methods. 

One might expect that the variables state initiative and state purposes, would be useful in 
discriminating between institutions that monitor the external impact on allocation or share of 
state funding and those that do not. Institutions with these state influences and purposes for 
collecting student assessment information would be expected to monitor the impact of that 
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Table 3. Results of Discriminant Analysis Showing Standardized Discriminant Coefficients' 





External Impacts 




Student 

Application 

or 

Acceptance 

Rates 


Allocation 
or Share of 
State 
Funding 2 


Evaluation 

from 

Regional 

Accrediting 

Agency 


Private 

Fund- 

Raising 

Results 


Success on 
Grant 
Applica- 
tions 


Commun 

-ications 

with 

External 

Constit- 

uents 


Institu- 
tional 
Reputation 
or Image 


Eigenvalue 


.092 


.207 


.114 


.091 


.102 


.151 


.182 


Wilks Lambda 


.915*** 


.829*** 


.897*** 


.917*** 


.907*** 


.869*** 


.846*** 


Functions At Group Centroids 
















Do Not Monitor (.00) 


-.149 


-.271 


-.384 


-.131 


-.181 


-.236 


-.307 


Monitor (1.00) 


.618 


.761 


.298 


.692 


.566 


.639 


.593 


External Influences 
















Accrediting Influence 




-.202 


.222 




-.275 






State Initiative 
State Approach 
State Authority 




.407 








.220 


.167 


Institutional Context 
Institutional Type 
Institutional Approach 




.381 












Academic Intentions 










.226 






Cognitive Assessment 
Affective Assessment 
Post-college Assessment 
Comprehensiveness of Data Collection 


.213 




.344 








.176 


Number of Instruments 




.242 








.260 


.245 


External Methods 




-.212 












Student Methods 








.401 








Number of Studies 


.502 


.235 




.406 


.391 


.466 


.253 


Number of Reports 
Institution-wide Support 




.206 


.268 










Mission Emphasis 
Internal Purposes 
Accreditation Purposes 
State Purpose 


.379 


.324 




.218 








Administrative and Faculty Support 




.214 












Breadth of Assessment Planning Group 
Number Approving Changes 


.250 


.188 












Management Policies and Practices 
















Access to Information 




.186 




.213 


.249 






Distribution of Reports 








.254 




.199 


.273 


Student Involvement 
Professional Development 
Student Affairs 


.210 


.229 


.388 




.318 


.250 


.170 


Faculty Evaluation 


.259 




.268 


• .339 


.223 


.251 


.271 


Academic Planning and Review 






.321 




.316 


.275 


.340 



Notes: 1. All coefficients shown are significant at p <, .001. 2. Analysis includes only public institutions. 

***p£.001 
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student assessment information. The contribution of institutional type would also be expected 
after examining the results of monitoring this impact by institutional type discussed in the 
previous section. These results generally show that as institutional type increases from associate 
of arts to research, institutional monitoring of external impacts increases. Of the two variables 
which negatively influence the discriminant function, accrediting influence might intuitively be 
expected since state funding and accreditation evaluation are generally viewed as separate and 
likely to have opposite effects in terms of monitoring influence. The negative influence of the 
variable external methods, a factor variable which consists of items relating to employer 
interviews and alumni interviews, shows that institutions using these type of student assessment 
methods are less likely to monitor external impacts of state funding. It is not clear why this 
might be the case, but perhaps external methods are used more frequently by private institutions 
who do not monitor the external impact on allocation or share of state funding. 

Institutional Reputation or Image (Table 3) is the external impact of student assessment 
that provides the second strongest discriminant function. The relative strength of the Wilks’ 
lambda (.846) and eigenvalue (.182) confirm this. The function at the group centroid (.593 and - 
.307), once again, significantly differentiates between the group that monitors versus the group 
that does not monitor. This function has eight discriminating variables that emerge from three of 
the five domains: external influences, institutional approach, and assessment management 
policies and practices. The two domains that do not contribute to the function are institutional 
context and institution-wide support. 

The strongest discriminating variable is academic planning and review (.340). This might 
be expected since institutions are likely to make programmatic changes in order to enhance their 
reputation or image. Distribution of reports is the second strongest discriminating variable. This 
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might be expected if institutions increase the distribution of student assessment information with 
the expectation that the distribution of this information will positively effect the institutions 
reputation or image. The third strongest discriminating variable is faculty evaluation. It is not 
immediately clear why this variable may influence the monitoring practices of institutions with 
regard to this external impact. Perhaps public institutions, which monitor this external impact 
more than private institutions, are more likely to use student assessment information in the 
faculty evaluation process. Additionally, institutions that use assessment information for faculty 
evaluation purposes, may expect that improved faculty evaluation processes will impact 
institutional image or reputation. 

Communications with External Constituent s (Table 3) is the impact of student assessment 
that provides the third strongest discriminant function, confirmed by the relative strength of the 
Wilks’ lambda (.869) and eigenvalue (.151). The function at the group centroid (.639 and -.236), 
also provide the third largest differentiation between groups. This function has seven 
discriminating variables which like the previous impact, emerge from only three of the five 
domains: external influences, institutional approach, and assessment management policies and 
practices. Again, the two domains that do not contribute to the function are institutional context 
and institution-wide support. 

The strongest discriminating variable is number of studies (.466). This perhaps makes 
intuitive sense that the more studies an institution undertakes, the more likely to report to 
external constituents and thus monitor the impacts of these communications. The second 
strongest discriminating variable is academic planning and review, which might be explained by 
institutions desire to communicate curricular and programmatic changes to external constituents 
and then monitor the subsequent impacts.,* The third strongest discriminating variable is number 
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of instruments. A logical explanation of this might be that the more instruments an institution 
uses the more student assessment information that is collected and thus the more likely to be 
communicated to external constituents. 

The Impact of Student Assessment on Evaluation from Regional Accrediting Agency 
(Table 3) provides the next best discriminant function. The relative strength of this function is 
corroborated by the Wilks’ lambda (.897) and the eigenvalue (.114). The function at the group 
centoid, however, shows the smallest difference between groups (.298 and -.384). This does not 
necessarily mean that the function is not good, but rather implies that there is not as great a 
difference between institutions that monitor this external impact versus those that do not. In this 
function there are only six discriminating variables that emerge from three of the five domains. 
Once again, the domains that are represented here include: external influences, institutional 
approach, and assessment management policies and practices. 

The strongest discriminating variable is student involvement (.388). This seems logical 
since accrediting agencies like to see student involvement in institutional processes. Two other 
variables that are nearly as strong are comprehensiveness of data collection (.344), and academic 
planning and review (.321). These both makes sense since accrediting agencies are generally 
positively influenced by more comprehensive student assessment efforts and since feedback 
from accrediting agencies often call for curricular or programmatic changes that effect academic 
planning and review. 

Success ofGraniApplicatipns (Table 3)provides the fifth strongest discriminant function of 
the seven external impacts of student assessment. The relative strength of the Wilks’ lambda 
(.907) and eigenvalue (.102) confirm this. The function at the group centroid (.566 and -.181), 
once again, significantly differentiates between the group that monitors versus the group that 



does not monitor. This function has seven discriminating variables that emerge from three of the 
five domains: external influences, institutional approach, and assessment management policies 
and practices. Again, the two domains that do not contribute to the function are institutional 
context and institution-wide support. 

The strongest discriminating variable is number of studies (.391). This variable, an 
additive index of the types of student assessment studies performed, would likely effect the 
monitoring of grant success, since the more studies an institution does the broader the scope of 
the information collected and the more likely it is to be used in grant applications. The second 
and third strongest discriminating variables are student affairs and academic planning and 
review, respectively. Similarly, these variables are likely to be influential in the overall 
discriminant function for this impact because certain grant application may ask for information 
that relates to student affairs function and the academic planning and review process. 

There is one discriminating variable in this function that contributes negatively to the 
function, accrediting influence (-275). This indicates that the more emphasis an institution 
places on the accrediting influence, the less likely it is to monitor the impact of success on grant 
applications. This may indicate that institutions that collect student assessment information 
mainly for accrediting purposes, are less likely to monitor the impact of that information on grant 
applications. 

Student Application or Acceptance Rates (Table 3) is the external impact of student 
assessment that provides the next to weakest discriminant function among the seven. The 
relative strength of this function is corroborated by the Wilks’ lambda (.915) and the eigenvalue 
(.092). The function at the group centoid (.618 and -.149) significantly differentiates between 
the group that monitors versus the group that does not. In this function there are six 
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discriminating variables that emerge from three of the five domains. The domains that are 
represented, however, are different and include: institutional approach, institution-wide support, 
and assessment management policies and practices. 

The strongest discriminating variable is number of studies (.502) which is the strongest 
influence of any of the discriminating variables for any of seven functions of the external 
impacts. This may suggest that institutions, which perform a number of different student 
assessment studies, are interested in increasing student application or acceptance rates and 
monitor this area in relation to the information they collect or the changes that they make as a 
result of the information they collect. One other variable that has a relatively strong influence is 
mission emphasis (.379). This is logical since institutions, which mention or emphasize student 
assessment in their mission, are likely to be interested in all aspects of student assessment 
information including the impact on student application or acceptance rates. 

Private Fund-raising Results, (Table 3) the external impact of student assessment variable 
that provides the weakest discriminant function, is confirmed by the relatively lower strength of 
the Wilks’ lambda (.917) and eigenvalue (.091) compared to the other six. The function at the 
group centroid, however, does provide the fourth largest differentiation between groups (.692 
and -.131). This function also has six discriminating variables which like the previous impact, 
emerge from three of the five domains: institutional approach, institution-wide support, and 
assessment management policies and practices. Again, the two domains that do not contribute to 
this function are external influence and institutional context. 

The two strongest discriminating variables, very close in their relative influence in the 
function, are number of studies (.406) and student methods (.401). These two variables probably 
have a strong influence in the overall discriminant function since private fund-raising methods 



are likely to include an emphasis to donors on the number and type of approaches to student 
assessment that the institution undertakes. 

Influential Discriminating Variables 

Having examined individually the variables discriminating among the seven external 
impacts, we now turn our attention toward an analysis of the five conceptual domains and 
individual variables, which contribute to the various discriminant functions. This will allow us 
to address more broadly our second research question: “How do institutions’ external influences 
on, institutional approaches to, institution-wide support for, and patterns of assessment 
management policies and practices for student assessment relate to the external impacts of 
student assessment information they monitor?”. We will do this by looking at which variables 
were influential across the seven external impacts. 

External Influences . Within the domain of external influences there were four possible 
discriminating variables: accreditation influence, state initiative, state approach, and state 
authority (Table 3). Of these, state approach did not emerge as a discriminating variable for any 
of the seven external impacts of student assessment. This suggests that regardless of the types of 
indicators or outcomes a state may mandate, it has no influence on the likelihood of an institution 
to monitor its external impacts of student assessment information. State initiative had a positive 
influence for two of the external impacts, allocation or share of state funding and 
communication with external constituents. The fourth variable, state authority, only emerged for 
one external impact, institutional reputation or image, and with a weak influence. Accrediting 
influence, however, emerged in three of the seven impacts: allocation or share of state funding, 
evaluation from regional accrediting agency, and success on grant applications. Interestingly, 
though in two of those functions, allocation or share of state funding and success on grant 



applications, it had a negative influence in the overall function. The likely reasons for this were 
previously discussed in the sections for each of these external impacts. 

The lack of number and pattern of state influences on the monitoring of external impacts of 
student assessment information may suggest that, apart from the monitoring of state funding, 
external influences from the state are not likely to provide much differentiating ability for 
distinguishing between institutions that monitor external impacts and those that do not. 
Conversely, external influences from accrediting agencies are likely to be useful in 
differentiating between institutions that monitor and those that do not. 

Institutional Context (Table 3). This domain has only one variable, institutional type, which 
was included in the discriminant analysis. Institutional control was not included in the analysis 
since it was a dichotomous (not continuous as required by discriminant analysis) variable. 
Though there are differences by institutional type in whether or not institutions monitor external 
impacts for three of the external impacts, student application or acceptance rates, allocation or 
share of state funding, and private fund-raising results, this variable only emerges as a 
significant discriminating variable for one of the seven external impacts, allocation or share of 
state funding. It does, however, have a relatively strong influence in the discriminant function of 
this external impact which suggests (as previously mentioned) that as institutional type varies 
from associate of arts to research, the institution is more likely to monitor the external impact of 
allocation or share of state funding. 

Institutional Approach . The domain of institutional approach is one of two domains that 
contribute a number of significant discriminating variables (Table 3). Of the ten approach 
variables used in the analysis, only three did not contribute any discriminating influence: 
cognitive assessment, affective assessment, and post-college assessment. Of the seven that were 



influential, there were three variables that were influential in three or more of the discriminating 
functions for the external impacts: comprehensiveness of data collection, number of instruments, 
and number of studies. The number of studies was the most influential variable in discriminating 
between institutions that monitor and those that do not, contributing a strong influence to six of 
the seven discriminant functions. 

The large number of variables in this section that emerged as influential across a number of 
external impacts suggests that the institutional approaches to student assessment, particularly in 
the areas of the number of studies, comprehensiveness of data collection and number of 
instruments, is useful in differentiating between institutions that monitor their external impacts 
and those that do not. 

Institution-wide Support . This domain, with seven variables used in the analysis, did not 
contribute any variable to a great extent (Table 3). Two of the variables, mission emphasis and 
number approving changes were influential in two of the discriminant functions: student 
application or acceptance rates and private fund-raising results (for mission emphasis) and 
student application or acceptance rates and allocation or share of state funding (for number 
approving changes). Overall, institution-wide support does not appear to be an area that 
differentiates between institutions that monitor external impacts and those that do not. 

Assessment Management Policies and Practices . The final domain of assessment 
management policies and practices appears to be the best domain for differentiating between 
institutions that monitor external impacts and those that do not. All seven variables in this 
conceptual domain contribute some influence to the discriminant functions for all seven external 
impacts (Table 3). Five of the seven contributed to three or more of the discriminating functions. 
Faculty evaluation contributed to six of the seven external impact functions and academi'c 



domains and dimensions of student assessment can be used to differentiate institutions with 
regard to the practice of monitoring. 

The descriptive results from this study were useful in addressing our first research question. 
The overall monitoring pattern of institutions is relatively low for all but one of the external 
impacts. Fewer than one-third of all institutions monitor the external impacts of their student 
assessment information with the exception being the external impact on evaluation from regional 
accrediting agency, where just over half of the institutions reported monitoring this external 
impact. There were differences in monitoring patterns that emerged based on the institutional 
type for three of the seven external impacts: student application or acceptance rates, allocation 
or share of state funding, and private fund-raising results. Additionally, differences also 
emerged based on institutional control. There were monitoring differences between public and 
private institutions for three of the seven external impacts: allocation or share of state funding, 
private fund-raising results, and institutional reputation or image. 

The discriminant analysis also proved useful in helping us address our second research 
question. We were able to identify which of the five conceptual domains are useful in 
differentiating between institutions that monitor external impacts of student assessment 
information and those that do not. Furthermore, the discriminant analysis helped in identifying 
specific dimensions within the conceptual domains that can be used to further differentiate 
between monitoring and non-monitoring institutions. 

Two of the five domains stand out as being very useful in differentiating between groups: 
institutional approach to student assessment and student assessment management policies and 
practices. These two domains contribute a number of variables to the discriminant functions of 
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planning and review contributing to four of the seven functions. They share four of the 
functions: evaluation from regional accrediting agency, success on grant applications, 
communication with external constituents, and institutional reputation or image. This may 
indicate a connection between the use of student assessment in faculty evaluation and for 
academic planning and review. 

It is clear that this domain is quite useful in differentiating between institutions that monitor 
their external impacts and those that do not. The use of student assessment information in 
faculty evaluation and for academic planning and review tend to be the most useful. 

Interestingly enough, however, previous studies (Peterson et al, 1999) have shown that 
institutions are least likely to use student assessment information in faculty evaluation in order to 
promote faculty involvement in student assessment 

Discussion and Implications 

The goal of this study is to help assess the extent to which institutions monitor the external 
impacts of student assessment information and to identify institutional domains and dimensions 
of activities supporting student assessment that will help institutions improve their assessment 
efforts. As a first step, it is helpful to know which of these domains and dimensions are actually 
used to help differentiate between institutions that are involved in monitoring certain impacts 
from student assessment data. While this study does not attempt to show a causal relationship 
between institutions that are involved in various assessment support practices and those that 
monitor external impacts, it does give insight into factors that can help differentiate between 
institutions that are involved (and possibly concerned with) the external impacts of their 
assessment information. Clearly this study has contributed to a better understanding of which 
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Interestingly, these findings (significant influence of institutional approaches and 
assessment management policies and practices and little influence of external influences, 
institutional context, and institution-wide support on monitoring the impacts of student 
assessment information) may coincide with a related paper on this topic. Peterson and Augustine 
(2000, forthcoming) find external influence, institutional context, and institution-wide support 
variables strongly related to the amount or extent of student assessment in which an institution is 
engaged. This suggests variables in these domains may be influential in getting an institution to 
initiate student assessment, but that approach and assessment management policies and practices 
are influential in fostering use and monitoring of the impacts of student assessment data. 

Future studies may wish to examine more intensively the nature of external impacts that 
result from the use of student assessment information and the quality of assessment efforts at the 
institutional level. It is clear from the literature that this is an area that has not been examined to 
any large degree. If the goal in this era of increased scrutiny is to improve institutional 
performance as well as student assessment activities, gaining a better understanding of 
assessment activities that help institutions improve their external impacts should be a high 
priority. Studies such as this and others can go along way toward achieving that goal. 
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